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Introduction 

The creators of 20/20 Analysis (also referred to herein as "20/20") propose that schools give special 
attention to students who demonstrate the least and most progress in basic academic subjects. This analysis 
focuses on one or more important outcomes of education and looks to the margins to identify students for 
whom the existing instructional program is working least and most adequately. It is based on the belief that 
schools that serve “marginal” pupils well will serve all students well (Reynolds, Zetlin, & Wang, 1993). 

20/20 is an outcomes-based approach to identifying and serving students at the margins and thus 
challenges the traditional programmatic divisions of narrowly framed categorical programs. These tradi- 
tional categories of student classification were established on the basis of presumed causes of learning 
problems that have been found to have very little relevance to instruction. As Reynolds and Zetlin (1993) 
note, “ [20/20] proposes that instead of organizing programs by categories based on presumed causes, schools 
proceed directly to outcome measures and to very intensive instruction for children whose progress in learn- 
ing under ordinary arrangements is marginal” (p. 3). 

Minneapolis Public Schools (MPS) became involved in 20/20 at the invitation of Margaret C. Wang 
of the National Center on Education in the Inner Cities (CEIC) at Temple University. Associate Superinten- 
dent for Research and Development, Jan Witthuhn, invited all interested MPS principals to a 20/20 presen- 
tation by Wang and Maynard C. Reynolds on February 20, 1991. Following this presentation, Dowling and 
Lincoln Elementary Schools volunteered to participate in a study on the effectiveness of 20/20. A 3-year 
collaborative effort among categorical service delivery staff, parents, administrators, and research personnel 
ensued. Data were gathered from central computer files and individual school records compiled according 
to the designs of the school staff. At Dowling, the initial data analysis and discussion were followed by 
cross-categorical restructuring of services for students identified through 20/20. 



Staff from Dowling decided to concentrate efforts on maximizing achievement based on the results 
of an analysis of the use of time by students and staff. As part of this analysis, they recommended that the 
time and effort devoted to categorical labeling as required by State of Minnesota special education eligibil- 
ity rules could be better spent in serving all students identified through 20/20 in a low-cost, noncategorical 
or cross-categorical model. Following discussions with parents and teachers, a draft of a waiver for perfor- 
mance” was presented by Dowling's principal, Jeff Raison, to Gary Warrington, head of the MPS Special 
Education Department. Mr. Warrington enthusiastically supported the request for a noncategorical waiver 
to the Minnesota Department of Education and proposed district-wide exception from state rules governing 
eligibility on learning disabilities and mild mental handicaps. This experimental proposal, approved in 
November 1993 by the Minnesota State Board of Education, highlights the 20/20 method in its “annual 
evaluation methods” section. The data included in each section of the following report serve as the baseline 
for monitoring the effects of the problem-solving method proposed here as an alternative to state eligibility 
rules. 

This paper provides additional detail and follow-up regarding Dowling's implementation of 20/20, 
as well as district-wide utilization of 20/20 in Minneapolis. It is organized into four sections: (I) an example 
of building-based 20/20 conducted at Dowling Elementary School; (II) a summary of statistics describing 
demographic and cross-categorical services for all MPS elementary schools; (HI) an explanation of continu- 
ous progress graphs (see Appendices A, B, and C) which illustrate the 20/20 approach to describing student 
progress in reading; and (IV) a discussion of 20/20's implications and possible uses. 

I. 20/20 at Michael Dowling Elementary School 
Obtaining Achievement Measures 

The first step in implementing 20/20 at Dowling was to define the outcomes of student learning on 
which 20/20 would be conducted. The Dowling staff decided to focus initial analysis on reading and 
mathematics. They opted to limit the analysis to scores for reading comprehension rather than vocabulary 
and word-attack skills. Similarly, measures of math facts computation were of less interest to the Dowling 
faculty than measures of quantitative concepts and applications. A problem arose in that students' test score 
data on the achievement measure utilized in this study, the California Achievement Test (CAT), were avail- 
able only for certain grade levels each year. Students who were enrolled as sixth graders in Fall 1991 were 
not tested on the CAT in Spring 1991 (i.e., as fifth graders), so it was necessary to use their fourth-grade 
scores from 1 990. 

CAT domain scores for reading comprehension and math concepts/applications were then down- 
loaded from the UNISYS mainframe for all students enrolled at Dowling as of October 1991. The reading 
comprehension and math concepts/applications scores from Spring 1991 were placed in rank order. It was 
then possible to find the point separating the bottom fifth (i.e., the 20th-percentile) and the top fifth (i.e., the 
SOth percentile) of the distribution. 
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For reading comprehension, the Dowling distribution closely resembled national norms but was, in 
fact, slightly below national norms at the 20th, 50th, and 80th percentiles in the total school distribution. 
The score which delineated the bottom fifth of all Dowling students who took the test corresponded to the 
18th percentile on national norms, while the median score and the score needed to place in the top fifth of the 
Dowling distribution corresponded to the 49th and 79th national percentiles, respectively. The complete 
distribution of Dowling reading comprehension scores for 1991 is presented in Figure 1.1 



Figure 1 

Spring 1991 Dowling Elementary 
Reading Comprehension Distribution 
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For the CAT math concepts/applications test, the score delineating the bottom fifth, the median 
score, and the score needed to place in the top fifth of the Dowling distribution corresponded to the 21st, 
56th, and 89th national percentiles, respectively. Figure 2 shows the entire Dowling math distribution for all 
students tested in 1991. 





Figure 2 

Spring 1991 Dowling Elementary 
Math Concepts/Applications Distribution 
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Once the cutoffs for the bottom and top fifths of the reading and math distributions were estab- 
lished, it was possible to examine the grade distribution and identify the students who scored in the upper 
and lower 20%. Since math and reading measures were used independently to determine the bottom- and 
top- 20% groups (also referred to herein as the "low-20" and "high-20" groups), it was also possible to 
determine the number identified by either or both procedures. Figure 3 depicts the low-20 group as identi- 
fied by the reading measure alone, math measure alone, or both. 

Through examining Figure 3, one notes that using two measures independently to determine the 
identity of students in the bottom 20% actually produces a combined total of 26.4% of the Dowling popula- 
tion as identified by either measure. 
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Figure 3 

Dowling Bottom 20% of Achievement by Area 
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Characteristics of Students at the Margins 

With the lower and upper fifths of the distribution established, our next step was to consider the 
demographic characteristics of the students in each group. Meiging the 20/20 achievement data with district 
codes for “free or reduced-price lunch” resulted in a breakdown of achievement related to socioeconomic 
status (SES). Table 1 summarizes the data for all Dowling students who took the CAT and had valid lunch 
codes in the district database (n=284). There is a high correlation between SES (as determined by participa- 
tion in free or reduced-price lunch programs) and reading achievement: two thirds (67%) of the students in 
the low-20 group were receiving free or reduced-price lunch. 

Table 1 

EaSSOiagSL of Students in 20/20 prouns b ased on CAT reading comprehensipn who rece ive 
fn»j» and reduced-mice lunch com pared to the total schpoi- 



Bottom 20% Top 20% Total school 



Percent free lunch 


58% 


10% 


31% 




Percent reduced-price lunch 


9% 


0% 


7 




Total percent free or 
reduced-price lunch 


67% 


10% 


38% 





In Figure 4, the Dowling students who scored in the bottom 20% of either the math or reading 
distributions are broken down by grade. One notes the overrepresentation of first- and fourth-grade students 

in this group. 



Figure 4 

Bottom 20% of Achievement in Reading or Math by Grade 
N=75 





In Figure 5a, the students in the bottom 20% of the reading distribution are broken down by racial/ 
ethnic group and then compared to the racial/ethnic breakdown of all students who took the CAT reading 
test in 1991 (see Figure 5b). Consideration of Figures 5a and 5b reveals that while Native- American stu- 
dents comprise only 1 3% of the total student population, they account for more than twice that percentage in 
the low-20 group (32%). Similarly, African-American students comprise 16% of the total population at 
Dowling School but 34% of the low-20 group. 

Figure 5a 

Bottom 20% of CAT Reading Distribution 
by Racial/Ethnic Group 
N=45 
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The disproportionate representation of African Americans and Native Americans in the bottom 20% 
of the Dowling reading comprehension distribution is similar to district-wide achievement results for stu- 
dents of color. Data from the 1990-91 school year indicate that while 32% of students enrolled in the district 
were African American, this group comprised 49% of those in the bottom quartile in reading. Similarly, 
5 1 % of all Chapter 1 students in 1 990-9 1 , and 44% of special education students in the district, were African 
American.^ 
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Figure 5b 

1991 Dowling Ethnic Distribution 
(for all 198 students who took the CAT) 
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Figure 6 shows the students in the top 20% of the reading distribution broken down by racial/ethnic 
group. Comparing this figure with Figure 5b evidences the underrepresentation of African-American and 
Native- American students in the upper fifth of the reading achievement distribution. 



Figure 6 

Top 20% of CAT Reading Distribution 
by Radical/Ethnic Group 
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Achievement Trends 

The next step was to examine trends in achievement across time for students continuously enrolled 
at Dowling. Data for students who were in third and fourth grades in 1992 were available from the CATs 
given in Spring 1990, 1991, and 1992 and were downloaded from the district's mainframe computer. These 
data were analyzed to determine shifts in the distribution during this 3-year period at the bottom-20% cutoff 
point, the median score, and the top-20% cutoff point. A summary of the change in reading achievement for 
first and second graders tested on the CAT reading comprehension test in 1990 is presented in Figure 7. 3 
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Figure 7 

DowlingCAX Reading Comprehension 1990-92 
Continuous Membership N=112 
(plot of NCEs with national percentile change noted) 
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One can see from Figure 7 that large shifts in the reading achievement distribution were accom- 
plished at each of the three points in the distribution. For example, in 1990 one fifth of students considered 
here were at or below the 12th national percentile in reading. By 1992, students who were enrolled continu- 
ously at Dowling showed an above average 27th national percentile at the cutoff for the bottom fifth of the 
school's distribution. Improvements were similar at the median, showing a rise from the 39th to the 58th 
national percentiles, and from the 79th to the 89th national percentiles for students at the cutoff for the top 
fifth of the school’s distribution. This last figure shows that a full 20% of Dowling’s continuously enrolled 
students were at or above the 89th percentile in 1992. 

A similar plot of the shift in achievement on the CAT math concepts/applications domain for stu- 
dents tested in 1990, 1991, and 1992 is presented in Figure 8 below. 
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Figure 8 

Dowling CAT Math Concepts 1990-92 
Continuous Membership N=112 
(plot of NCEs with national percentile change noted) 




Again, the average (median) students or students near the bottom of the math distribution were not 
the only ones who made gains within the collaborative model at Dowling. The students near the top of the 
achievement distribution made gains similar to students near the median.^ Gains of over 10 Normal Curve 
Equivalents (NCEs) for the median and top 20% of the distribution are quite significant; in fact, Slavin and 
Madden (1989) have indicated that NCE changes of 1/3 of a standard deviation (or about 7 NCEs) could be 
used as a benchmark for program effectiveness. 

These achievement results were presented to the principal and leadership team at Dowling in the fall 
of the 1992-93 school year. The team was very encouraged by the achievement trends and suggested certain 
follow-up analyses be conducted to investigate whether similar achievement gains had been made by each 
of the racial/ethnic groups represented at Dowling. In order to conduct this analysis, data from all students 
tested at Dowling in 1990 were compared against the scores for the same students in 1992. This 2-year 
analysis included students tested in fourth grade in 1990 and in sixth grade in 1992 who were not included in 
the 3-year analysis.^ 

Figure 9 depicts the 2-year reading comprehension trend for African-American, Native- American, 
and white students who were enrolled at Dowling and tested using the CAT in 1990 and 1992. The median 
of the distribution for each ethnic group is plotted for both periods. For example, the lOth-ranked Native- 
American student (i.e., out of 19 continuously enrolled Native Americans at Dowling) scored at the 12th 
percentile on the CAT in reading in 1990, whereas the lOth-ranked Native- American student in 1992 scored 
at the 31st percentile. 
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Figure 9 

Dowling Gains in CAT Comprehension by Racial/Ethnic Category 
Continuous Membership N=141 
(plot of NCEs with national percentile change noted) 




As one can see from Figures 9 and 10, all three racial/ethnic groups demonstrated a positive trend in 
median scores for reading and math from 1990 to 1992. The gain in the median score for African-American 
students was relatively less than the gain for Native Americans in reading but was quite similar in math. 
This type of trend analysis is central to the 20/20 process, which is most concerned with rates of progress for 
individuals and groups. 



Figure 10 

Dowling Gains in CAT Math Concepts by Racial/Ethnic Category 
Continuous Membership N=141 
(plot of NCEs with national percentile change noted) 
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It is noteworthy that students were not separated physically by racial/ethnic groups in this analysis, 
but that it was possible to disaggregate data to show what progress was being made by various subgroups 
under the conditions then prevailing at the school. 

Services for Students in the Bottom 20% 

The next step focused on the services being provided to individual students in the school. Eight 
different instructional interventions/categories were identified and coded for Dowling students: social work 
groups. Higher Order Thinking (HOTS), mentoring, Native-American support, and gifted/talented, special 
education. Chapter 1, and collaborative services. Many students were identified as receiving multiple ser- 
vices while some students were not receiving any. 

The complete listing of services and 20/20 groups for math and reading were presented to the build- 
ing leadership. Table 2 reports only the primary categories of special education. Chapter 1, and collabora- 
tive services. For simplification of the table, if a student was receiving more than one of these services, the 
service listed to the left on the table was coded. For example, a student receiving both Chapter 1 and special 
education services was coded “Special,” and a student in a collaborative services group who was also la- 
beled Chapter 1 was coded “Chapter 1.” Students listed under “Collaborative” were receiving that single 
service. 

Table 2 

Number of students receiving categorical services b v the 20th-percentile cutoff s. 





Special 


Chapter 1 


Collaborative 


Noncategorical 


Total 


Below 20% on reading only 


6 


9 


5 


19 


39 


Below 20% on math only 


2 


9 


3 


8 


22 


Below 20% on both 


16 


19 


3 


2 


40 


Above 20% on both 


0 


19 


15 


176 


210 


TOTAL 


24 


56 


26 


205 


311 



One can see from Table 2 that all (100%) of the Dowling special education students tested in 1991 
were below the 20th-percentile in reading, math, or both areas. Thus, if a 20th-percentile cutoff (math and 
reading) had been used to offer special education services to students, not a single student who was actually 
classified to receive these services at the Dowling School in 1991 would have been omitted. Approximately 
two thirds (37/56=66%) of the Chapter 1 students and a little more than half (8/15=53%) of the collaborative 
service students scored below the 20th-percentile cutoff. 
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Cross-Validation with Curriculum-Based Measures 

In Fall 1992, 20/20 at Dowling was repeated using curriculum-based measures (CBM) of reading 
(Deno, 1985). Oral reading from basal and literature passages provided a direct measure of reading for 
contrast and correlation with the CAT reading comprehension subtest scores. CBM scores were referenced 
to local norms and then converted to building percentiles in the same manner as the CAT scores from Spring 
1992. Scores from grades 2-5 in CAT reading comprehension were then merged with CBM scores to deter- 
mine the standard score correlation and degree of agreement in determining the 20/20 groups. 

The standard score correlation (i.e., using NCEs) between the CAT reading comprehension and 
CBM Words Correct measures was .75 - a very high validity coefficient given the fact that the two tests 
were administered 5 months apart. The scatterplot shown in Figure 11 shows that most points in the bivari- 
ate distribution lie close to the regression line. There are some outlying students who scored low on one test 
and in the average range on the other, but the agreement on classifying students in the low- and high-20 
groups is substantial. 

Figure 11 

Correlation between CAT Reading Comprehension 
and CBM Words Correct (Dowling Grades 2-5) 
y = 7.8532 + 0.86585x R=.75 




Agreement in classifying the 20/20 groups is presented in Table 3. The agreement for the top fifth 
of the distribution was 76% (26/34) and for the bottom fifth, 65% (22/34); overall the classification agree- 
ment was 76% (133/175). 
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Table 3 

Agreement in 20/20 classification between the CBM and CAT reading 
comprehension measures . 





Bottom 1/5 
CBM 


Middle group 
CBM 


Top 1/5 
CBM 


Total 


Percent 

agreement 


Bottom 1/5 CAT 


22 


12 


0 


34 


65% 


CAT middle group 


9 


85 


13 


107 


79% 


Top 1/5 CAT 


0 


8 


26 


34 


73% 


TOTAL 


31 


105 


39 


175 


76% 



With the information provided in the comparison of CBM and CAT reading comprehension scores, 
the staff at Dowling are currently assessing the costs and benefits of using one procedure over the other. In 
general, any form of authentic or performance-based assessment which can be put on a rank order scale can 
potentially be used in 20/20. As Reynolds and Zetlin (1993) indicate in A Manual for 20/20 Analysis, 

In [20/20] one simply starts with whatever assessment procedures are in use in the schools being 
studied. It is assumed that there will always be efforts to improve procedures for testing and deci- 
sion making in the schools. This plan proposes that educators doing 20/20 analyses should join in 
efforts to improve assessment procedures, and be ready to use new approaches as their validity is 
established. ... In summary, one starts with the assessments in hand . . . but then helps to find better 
approaches as rapidly as possible, (pp. 10-11) 

II. District- Wide 20/20 
Demographic and Service Summary Data 

District-wide 20/20 Analysis for all elementary schools (n=50) was performed using Spring 1992 
CAT reading comprehension tests. The cutoff points for the top and bottom fifths of the reading distribu- 
tions were obtained for students in all 50 schools, including all special-needs students. For students who 
were not tested in Spring 1992, the test score for the preceding school year was used. For example, since 
fifth graders were not tested in 1992 with the CAT, their fourth-grade scores from Spring 1991 were used. 

Each student enrolled in MPS elementary schools in Spring 1992 was assigned a score for reading 
comprehension. Moderately and severely handicapped special education students who were not tested were 
assigned the 1st percentile while mildly handicapped students were assigned the percentile of their most 
recent testing. For some students who were not tested prior to 1992, the reading comprehension score for 
Spring 1993 was used as the best estimate of performance for 20/20. 

Once achievement cutoffs were established, the analysis was conducted according to the same pro- 
cedures as those used for Dowling Elementary. Percentages of handicapped students captured by the bottom 
one-fifth cutoff were tallied by disability designation for each school. Similarly, the percentages of Chapter 
1, Limited English Proficiency (LEP), and gifted/talented students falling within the bottom and top fifths of 



the distribution were calculated. 20/20 groups were contrasted on racial/ethnic status, free or reduced-price 
lunch status, “resides with” codes, and gender. The tables and figures contained in this section summarize 

the overall findings. 

The average cutoff score for the bottom fifth of the achievement distribution for all elementary 
schools combined was the 11th percentile nationally on the CAT reading comprehension measure. The 
distribution of scores for all students with test scores is presented in Figure 12. If data for MPS elementary 
schools were exactly comparable to data for students nationally, the bars comprising the graph (Figure 12) 
would be flat or even. In fact, they show elevation to the low (left) side of the figure, indicating an excess of 
low scores in reading for MPS students as compared with national norms which, despite the fact that the data 
show a distribution almost at national average levels near the top of the scale, is most notable. 



Figure 12 

MPS Elementary Reading Comprehension Distribution 
Tbtal Studecns Tfested = 25,112 
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20/20 could be performed on the distribution of scores for the whole district (see Table 6), but since 
it is designed for local school initiative, it is more appropriate to identify cutoff points individually for each 
school, perform separate analyses based on the actual cutoffs for each school, and then aggregate the total 
district data based on these analyses. 

Individual school analysis was conducted on all elementary schools using the 1992 CAT reading 
comprehension scores along with information related to LEP and special education caseloads. Every stu- 
dent enrolled in grades 1-6 was entered into the analysis for each school. The achievement distributions 
varied greatly from school to school as can be seen from the distribution of bottom-fifth cutoffs shown in 
Figure 13. 

In Figure 13, one can see the very large differences in overall achievement levels among MPS 
elementary schools. The bottom-fifth cutoff ranged from the 1st percentile on national CAT norms in two 
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schools to the 35th percentile in one school. Similarly, although not graphed here, the distribution of cutoffs 
for the top fifth of the reading comprehension distribution ranged from the 39th to the 91st percentile. 



Figure 13 

CAT Reading Comprehension Bottom-20th 
Percentile Cutoff by School 
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Demographic breakdowns of the bottom and top fifths of the distributions were calculated for each 
school using the same method employed for Dowling Elementary. A summary of these demographic con- 
trasts is presented in Table 4a below. 
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Table 4a 

Percentages of students in 20/20 groups (re a ding^ who are receiving free QL redu c e d jqce lunch. living 
with single parents, or students of color compared to distric t tota l elemeo tarY .. PCtcettfages . 





Bottom 20% 


Top 20% 


Total elementary 


Percent free or reduced-price lunch 


76% 


33% 


56% 


Percent living with single parents 


61% 


31% 


49% 


Percent students of color 


75% 


30% 


55% 



For the total population of MPS elementary school students, 56% received free or reduced-price 
lunch. If there were no correlations with low-20 and high-20 groups, the figure would be about 56% in each 
group. However, as shown in Table 4a, 76% of students in the low-20 group received free or reduced-price 
lunch whereas this figure was only 31% for students in the high-20 group. Data from Table 4a also show 
disproportionalities with reference to “percent living with single parents” and “percent students of color.” 
20/20 can also be used to investigate relationships among demographic variables. For example. 
Table 4b reports data for all grade 1-3 students in the low-20 group by free or reduced-price lunch and racial/ 
ethnic codes. These data show the strong relationship between indicators of poverty and racial/ethnic status 
for marginally achieving students. Students of color in the bottom 20% of the district reading achievement 
distribution ranged from 80% to 92% free or reduced-price lunch status. White students also exhibited 
higher proportions of free or reduced-price lunch status in the low-20 group (56%). 



Table 4b 

Relationship between free or reduced-price lunch and racial/ethnic status 
for students in the bottom-20% group (reading) . 





Free or 

reduced-price lunch 


Full-price lunch 


Percent free or 
reduced-price lunch 


African American 


1380 


221 


86% 


Asian 


396 


34 


92% 


Native American 


317 


76 


81% 


Hispanic 


91 


23 


80% 


White 


479 


376 


56% 


TOTAL 


2663 


730 


79% 
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20/20 cutoffs were also tabulated for various categorical programs and student disability labels. 
Table 5 summarizes the 20/20 groups for students served in special education. Chapter 1, LEP, and gifted/ 
talented programs as identified in the district mainframe computer codes in December 1992. 

Table 5 indicates the percentages of students in each 20/20 group receiving services from various 
categorical programs. For example, 26% of the students in low-20 groups on reading achievement, accord- 
ing to individual school cutoffs, received special education services under the primary handicapping condi- 
tion of learning disabled. Overall, 36% of the students in the bottom fifths of their respective school distri- 
butions were receiving special education services in 1992. Similarly, 26% of students in low-20 groups 
received Chapter I services, and 23% received LEP services. The unduplicated total of students receiving 
categorical services was 66%; that is, about two thirds of the students in the bottom fifths of their schools in 
reading were receiving some form of categorical service.^ The aggregate of all data on the top fifths of each 
school’s reading distribution revealed that 30% of these high-20 students were being served in gifted/tal- 
ented programs. 



Table 5 

Percentages of students in 20/20 groups (reading) in c ategorical Programs. 





Bottom 20% 


Top 20% 


Total elementary 


Special education: 


26% 


.3% 


7% 


Learning disabled 






Special education: 


4% 


0% 


1% 


Mentally handicapped 






Special education: 
E/BD 


2% 


.2% 


1% 


Special education: 


3% 


1% 


2% 


Speech and language 






Special education: 


.4% 


0% 


.2% 


Physically handicapped 






Special education: 


.6% 


0% 


.2% 


Other handicaps 






Special education: 


36% 


.3% 


12% 


All disabilities 


2% 


15% 


26% 


Chapter 1 


1% 


10% 


23% 


LEP 






Total categorical: rcmedial/spccial 


66% 


4% 


30% 


education (unduplicatcd) 








2% 


30% 


12% 



Gifted/talented 
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The data contained in Table 6 are presented on a district-wide basis. Compared with Table 5, in 
which all students are grouped according to their presence in low- or high-20 groups in their individual 
schools. Table 6 shows that an even larger percentage of low- 20 students in reading receive some form of 
categorical service. This table indicates that approximately 85% (as opposed to 66% on Table 5) of the 
students who scored below the district- wide 20th-percentile cutoff in reading (the 1 1th percentile nationally) 
were receiving special education. Chapter 1, or LEP services. 

Although Table 6 demonstrates that 20/20 can readily be utilized on a district-wide basis and fo- 
cused on specific demographic characteristics (in this case, students in grades 1-3), a key component of 
20/20 is to get local school staff interested in examining students who are making less than adequate aca- 
demic progress but who are not receiving individual attention from specialists in the building. As building 
staff look closely at those students at the margins they not only identify factors which the school cannot 
modify and which are correlated with achievement (such as poverty and single-parent families), but they 
also more productively attend to variables which can be manipulated by teachers and which have been 
shown to impact achievement. This sort of school-wide focus may lead to a reorganization of categorical 
services in a more coordinated and efficient manner to reach all students at the margins. Certain critical 
variables which affect academic performance for all students — such as flexible pacing and increased time 
on task — may become the focus of collaborative efforts among regular teachers, specialists, parents, and 
administrators. It is the ultimate goal of 20/20 to first identify individual students who need such adapta- 
tions and then empower all individuals responsible for student achievement to focus on instructional ar- 
rangements that have the greatest chance of maximizing student performance. 



Table 6 



Number and percentage of students in grades 1-3 who scored below the district- wide zuth-percentue (reading! 
cutoff (i.e.. the 1 1th national percentile) served in special education. Chapter 1. and Limited English ProflcieJlCY 


(LEP) programs. 


Category 


Bottom 20% 


Special education: Learning disabled 


703 


Special education: Mentally handicapped 


16 


Special education: E/BD 


56 


Special education: Speech and language 


127 


Special education: Physically handicapped 


12 


Special education: Other handicaps 


9 


Special education: All disabilities 


923 


Chapter 1 


1408 


LEP 


328 


Total categorical services 


2659 


Total students in bottom- 20% group 


3134 


Percent of bottom-20^ group served 


84 . 8 % 
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III. Continuous Progress Graphs Illustrating Student Progress in Reading 

The continuous membership database of all students enrolled for 3 years and tested using CAT 
provides an opportunity for considering not only the average performance for a school (e.g., the median 
score, which represents the middle score in a school's distribution) but also for examining the margins to see 
that students at or below the bottom-fifth cutoff and at or above the top-fifth cutoff are making adequate 
achievement gains. The reading graphs presented in the appendices depict progress first for all grades K-6 
and K-8 MPS schools (Appendix A, Figures 15-41); then for grades K-3 schools (Appendix B, Figures 42- 
53); and finally for grades 4-6 schools (Appendix C, Figures 54-60). The K-6 and K-8 MPS school graphs 
include progress data for three grades across 3 years (i.e., the first-, second-, and fourth-grade students in 
1990 who are enrolled in the same school 2 years later in third, fourth, and sixth grades). 

How to Read these Graphs 

Each graph is based on scores from all students who were enrolled and tested all 3 years in the same 
school.' 

Figure 14 shows the progress trends for all students in K-6 or K-8 schools from 1990-92 (n=4,589) 
and is used as an example here for pointing out the key features for understanding the figures presented in 
the appendices. 

These graphs depict progress using NCE scores. Scaled from 1 to 99, NCE scores are similar to 
percentile scores, yet have the advantage of being calibrated on an equal-interval scale, which allows data 
from several grades to be legitimately aggregated and reported. Furthermore, this scale allows comparison 
of progress at the top- and bottom-fifth cutoffs of the distribution. 

On the NCE scale, the 1st percentile equals 1 NCE, the 50th percentile equals 50 NCEs, and the 
99th percentile equals 99 NCEs. However, these are the only points of correspondence. One can see from 
Figure 14 that the 80th percentile equals 68 NCEs, while the 20th-percentile equals 32 NCEs. The 20th, 
50th, and 80th national percentiles are indicated by solid lines. For schools whose distributions are vastly 
lower than national distributions, these lines are omitted. 

Figure 14 shows that students in K-6 and K-8 schools have made progress in their standing on 
national norms at the 20th-percentile cutoff point, the median, and the 80th-percentile cutoff point. When 
gains are seen at all points in the distribution, it can be stated clearly that the MPS elementary school 
population as a whole is making progress. Some of the graphs in this section show no gains at all, while 
others show gains for the bottoms and tops of the distributions only. 

The presentation of data in this section is descriptive, not inferential. That is, these data are not 
appropriate for hypothesis testing such as, “Are the gains from school A significantly different from school 
B for the bottom group?” Other types of experimental procedures would be needed to perform such analy- 
ses. These graphs are presented for the general purpose of stimulating interest in the measurement of stu- 
dent progress trends and specifically for examining achievement trends for students at the margins. 

As in Figure K, which represents the entire district, most of the individual school graphs show 



substantial gains at both margins in reference to national norms for reading comprehension. Again, these 
gains reflect the progress of students continuously enrolled. In Figure 14, these gains represent the 4,589 
students who were enrolled and tested in the district for all 3 years; the gains on the individual school graphs 
represent students enrolled and tested in given schools for all 3 years. 



Figure 14 

MPS K-6 & K-8 Elementary Schools CAT Reading Comprehension 1990-92 
Continuous Membership N=4,589 
(plot of NCEs with national percentile change noted) 
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The data for individual schools are presented below without supplementary narrative explanations. 
However, brief comment may be useful in connection with the first graph, which represents Andersen Open 
School (Figure 15). It may be observed that data for 1990 showed that pupils at Andersen scored somewhat 
below average in reading compared to the nation's children in general (the 20th, 50th, and 80th school 
percentiles at Andersen corresponded with the 9th, 26th, and 62nd national percentiles). In the 2-year period 
from 1990-92, the data show an advance in reading achievement, relative to national norms, on all three 
levels. Although these data are only for students who were continuously enrolled at Andersen, it may be fair 
to conclude, at least tentatively, that positive changes were occurring for students at the school over this 
period. 
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Figure 15 

Andersen Open CAT Reading Comprehension 1990-92 
Continuous Membership N=176 
(plot of NCEs with national percentile change noted) 
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IV. Implications and Possible Uses of the Procedure 
It is possible to generate several types of data with 20/20 Analysis. The procedures are simple and 
low cost The data are easily and quickly understandable by all stakeholders in the school situation, most 
significantly by parents. Some potential uses of the procedure are explained briefly below. 

Allocation of Students to Special Support Services 

A key recommendation of the developers of 20/20 is that schools might use the procedure in select- 
ing students to receive especially intensive instruction on basic elements of the curriculum. As shown in 
data for MPS schools grades K-3, 85% of students in low-20 groups are already in categorical programs. We 
suggest establishing— as policy— that whenever a student falls into the lower fifth of his or her class in rate 
of progress in basic skills, his or her parents be notified and a conference be held to advance ideas and plans 
that promise better results. No labels are involved, only the basic acknowledgment of a need to do better. A 
simplified Individualized Educational Plan (IEP) format has been designed which could be used in all low- 
20 cases. It meets federal requirements but is simplified in several respects. This format includes a brief 
section for recording commitments parents are willing to make to enhance the student s life circumstances, 
in addition to detailed plans for instruction and management in school. As soon as feasible, we would 
propose that the commitment to collaboration with parents and individualized planning be extended to top- 
20 students as well. Under this allocation plan, parents and teachers also would be permitted to refer non- 
20/20 students for special diagnostic studies and programmatic adaptations. For example, students with 
hearing or vision problems might not fall into 20/20 groups but still require adapted instructional programs. 
Similarly, students showing extreme emotional problems would well require special accommodations even 
if their achievement records were adequate. 



Redeployment of School Psychologists 

The allocation plan suggested above would result in opportunities to redeploy psychologists from 
simple psychometric and classification functions and permit them to become engaged in functions more 
closely related to instructional improvements. For example, psychologists might be expected to provide 
leadership to programs relating to uses of time (in school and/or at home), as well as advancing practices 
related to student resilience, metacognition, and social behavior. 

New Approaches to Accountability and Program Evaluation 

The 20/20 procedure produces data revealing directly and immediately how well schools are doing 
in causing learning. Although the procedure would require careful interpretations, the data have a great deal 
of face validity in showing whether individual schools are exhibiting progress in students’ learning out- 
comes. 

New Approaches to Allocation of Resources 

Data included in this report show enormous differences among schools in the number of students 
who are behind in their learning. There are schools in the present study in which 20% or more of the pupils 
rank at or below the 5th percentile on national norms. Clearly, changes are needed in such school situations 
to provide more intensive help to students whose progress is minimal. The 20/20 procedure may be espe- 
cially useful in such cases because it looks to the margins, not only to means or medians. Recent discussions 
about how efforts to revise Chapter 1 programs have the apparent effect of concentrating special resources 
in schools that serve relatively large populations of poor children and those showing least progress in learn- 
ing are relevant to what is proposed here — using data focused on the margins as a basis for allocating 
compensatory and remedial resources. 

Disaggregation of Data on School Learning 

Having data available in centralized computer files makes it very simple to disaggregate data for a 
variety of purposes. For example: 

• Data for successive grade levels can show that a given school might receive pupils of relatively low 
standing in early grades but who show consistent gains as they advance through successive grades. 
Examples of such analyses are shown in this report. When there is evidence of such improvements, 
there is reason to continue support to programs that appear to be producing desired results. 

• Data can be disaggregated to show for a given school or district as a whole how students of various 
racial/ethnic, socioeconomic, or gender groups are progressing. Therefore, without segregating groups 
of students in any way, we can show how children of poor families, for example, are performing 
under various conditions in school. Data which reveal such results can thus be used in school plan- 
ning and in collaborative work with parents to reach desired improvements. 

• Data can be disaggregated to show the achievement records of pupils who are continuously enrolled 
compared with those who change schools frequently. Such data can be useful in setting policies that 
influence continuity in school placements. 
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Conclusion 

20/20 is proposed as a simple, low-cost, and efficient procedure for analyzing students’ progress 
and evaluating programs provided for them. It looks to the margins, as well as to central tendencies, in 
assembling data for school planning. Potentially, 20/20 can assist in diminishing classifications and the 
sometimes demeaning — labeling of students, and help in creating inclusive and powerful schools that serve 
our increasingly diverse student population. It offers ways of observing how various subpopulations of 
students (by race, ethnicity, socioeconomic level, etc.) are performing in schools through aggregates of data, 
not by forming special homogeneous groups for instruction. The procedure can be helpful in redeployment 
of some specialists, such as school psychologists, in ways that foster instructional improvements based on 
well-confirmed knowledge about human learning. Experience, so far, is that 20/20 procedures are readily 
understood by parents and can provide the basis for increased school-home collaboration. 20/20 Analysis 
offers a dynamic alternative to the disjointedness of numerous narrowly formed categorical programs, which 
are evaluated mainly in procedural terms, and opens opportunities for creative approaches to instruction for 
all students. 
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Endnotes 



1 If the Dowling distribution were equivalent to national norms at all points in the distribution, there 
would be a rectangular shape to the graph with 10% of all students in the school scoring at each 
decile. 

2 These data are summarized in detail in Disproportionate Representation in Special Education , an 
unpublished evaluation paper presented to the MPS Superintendent’s Cabinet, March 25, 1991. 

3 It should be noted that the change in the reading achievement distribution from 1990 to 1992 de- 
picted in Figure 7 is plotted using Normal Curve Equivalents (NCEs), an equal interval metric. 
Plots of percentile units tend to inflate gains near the median and underrepresent gains near the 
margins of the distribution. Percentile points are noted on the graph for familiar reference but 
should not be subtracted to compare gains at different points in the distribution. 

4 For technically sophisticated readers, the results of the analysis presented here are not affected by 
the problem of “regression to the mean.” The 20% and 80% cutoffs reflect shifts in the entire 
distributions rather than parameters of groups selected on the basis of extreme scores. 

5 Because no fifth graders were tested with the CAT, it was not possible to include these students in 
the 3-year analysis depicted in Figures 7 and 8. 

6 Since some students receive multiple services (e.g., Chapter 1 and special education) the total of 
LEP, special education, and Chapter 1 percentages add up to more than the unduplicated total. 

7 This deliberately includes special education and LEP program students who are sometimes ex- 
cluded from MPS’s district-wide testing summaries. 20/20, with its focus on students at the mar- 
gins, always includes every student in the building for whom an appropriate outcome measure can 
be found. 
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Appendix A 

Reading Graphs for Grades K-6 and K-8 Schools 

1990-92 



Figure 16 

Bethune CAT Reading Comprehension 1990-92 
Continuous Membership N= 48 
(plot of NCEs with national percentile noted) 
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Figure 17 

Bethune Public School Academy (PSA) 

CAT Reading Comprehension 1990-92 
Continuous Membership N=60 
(plot of NCEs with national percentile change noted) 
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Figure 18 

Burroughs CAT Reading Comprehension 1990-92 
Continuous Membership N= 177 
(plot of NCEs with national percentile change noted) 
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Figure 19 

Dowling CAT Reading Comprehension 1990-92 
Continuous Membership N=144 
(plot of NCEs with national percentile change noted) 
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Figure 20 

Fulton CAT Reading Comprehension 1990-92 
Continuous Membership N=188 
(plot of NCEs with national percentile change noted) 
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Figure 21 

Hall CAT Reading Comprehension 1990-92 
Continuous Membership N= 95 
(plot of NCEs with national percentile change noted) 
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Figure 22 

Hamilton CAT Reading Comprehension 1990-92 
Continuous Membership N=60 
(plot of NCEs with national percentile change noted) 
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Figure 23 

Jefferson CAT Reading Comprehension 1990-92 
Continuous Membership N=128 
(plot of NCEs with national percentile change noted) 
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Figure 24 

Kenwood CAT Reading Comprehension 1990-92 
Continuous Membership N=169 
(plot of NCEs with national percentile change noted) 
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Figure 25 

Lincoln CAT Reading Comprehension 1990-92 
Continuous Membership N=206 
(plot of NCEs with national percentile change noted) 
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Figure 26 

Marcy CAT Reading Comprehension 1990-92 
Continuous Membership N= 104 
(plot of NCEs with national percentile change noted) 
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Olson CAT Reading Comprehension 1990-92 
Continuous Membership N= 224 
(plot of NCEs with national percentile change noted) 
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Figure 28 

Barton CAT Reading Comprehension 1990-92 
Continuous Membership N=114 
(plot of NCEs with national percentile change noted) 

90q 



-SoT 70 



2 
c 
a* 

> On 

"3 o 
cr~ 

g s 

£: £ 
3 *— 
(J 

*c3 
a u 

EQ 

u 

o 

Z 



60- 



50 



40 



: 88lh percentile 


9 1 st percentile 




70th percentile 


66th percentile 




O 

■ 42nd percentile 


O 

41st percentile 








1989 1990 

Grades 1,2,4 



Year 



1992 

Grades 3,4,6 



1993 



Figure 29 

Putnam CAT Reading Comprehension 1990-92 
Continuous Membership N=42 
(plot of NCEs with national percentile change noted) 
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Figure 30 

Seward Montessori CAT Reading Comprehension 1990-92 
Continuous Membership N=118 
(plot of NCEs with national percentile change noted) 
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Figure 31 

Sheridan CAT Reading Comprehension 1990-92 
Continuous Membership N=101 
(plot of NCEs with national percentile change noted) 
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Figure 32 

Ramsey Fine Arts Reading Comprehension 1990-92 
Continuous Membership N=179 
(plot of NCEs with national percentile change noted) 
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Figure 33 

TUttle Reading Comprehension 1990-92 
Continuous Membership N=95 
(plot of NCEs with national percentile change noted) 
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Figure 34 

Webster CAT Reading Comprehension 1990-92 
Continuous Membership N=154 
(plot of NCEs with national percentile change noted) 
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Figure 35 

Willard M/S/T Reading Comprehension 1990-92 
Continuous Membership N=170 
(plot of NCEs with national percentile change noted) 




Figure 36 

Windom CAT Reading Comprehension 1990-92 
Continuous Membership N=110 
(plot of NCEs with national percentile change noted) 
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Figure 37 

Wilder Fundamentals CAT Reading Comprehension 1990-92 
Continuous Membership N=1 65 
(plot of NCEs with national percentile change noted) 




Figure 38 

Wilder MJSJT CAT Reading Comprehension 1990-92 
Continuous Membership N=65 
(plot of NCEs with national percentile change noted) 
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Figure 39 

Marcy CAT Reading Comprehension 1990-92 
Continuous Membership N= 104 
(plot of NCEs with national percentile change noted) 
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Figure 40 

Wilder M/S/T CAT Reading Comprehension 1990-92 
Continuous Membership N=65 
(plot of NCEs with national percentile change noted) 





Figure 41 

Four Winds CAT Reading Comprehension 1990-92 
Continuous Membership N=99 
(plot of NCEs with national percentile change noted) 
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Appendix B 

Reading Graphs for Grades K-3 Schools 

1990-92 



Figure 42 

MPS K-3 Elementary Schools CAT Reading Comprehension 1990-92 
Continuous Membership N= 1,841 
(plot of NCEs with national percentile change noted) 
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Figure 43 

Armatage CAT Reading Comprehension 1990-92 
Continuous Membership N=79 
(plot of NCEs with national percentile change noted) 
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Figure 44 

Cooper CAT Reading Comprehension 1990-92 
Continuous Membership N=44 
(plot of NCEs with national percentile change noted) 
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Figure 45 

Hale CAT Reading Comprehension 1990-92 
Continuous Membership N=154 
(plot of NCEs with national percentile change noted) 
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Figure 46 



Hiawatha CAT Reading Comprehension 1990-92 
Continuous Membership N=50 
(plot of NCEs with national percentile change noted) 
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Figure 47 

Howe CAT Reading Comprehension 1990-92 
Continuous Membership N=54 
(plot of NCEs with national percentile change noted) 
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Figure 48 

Keewaydin CAT Reading Comprehension 1990-92 
Continuous Membership N=49 
(plot of NCEs with national percentile change noted) 
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Figure 49 

Kenny CAT Reading Comprehension 1990-92 
Continuous Membership N=68 
(plot of NCEs with national percentile change noted) 
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Figure 50 

Loring CAT Reading Comprehension 1990-92 
Continuous Membership N=57 
(plot of NCEs with national percentile change noted) 
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Figure 51 

Morris Park CAT Reading Comprehension 1990-92 
Continuous Membership N=69 
(plot of NCEs with national percentile change noted) 
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Figure 52 

North Star CAT Reading Comprehension 1990-92 
Continuous Membership N=58 
(plot of NCEs with national percentile change noted) 
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Figure 53 

Wenonah CAT Reading Comprehension 1990-92 
Continuous Membership N=54 
(plot of NCEs with national percentile change noted) 
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Appendix C 

Reading Graphs for Grades 4-6 Schools 

1990-92 



Figure 54 

MPS 4-6 Elementary School CAT Reading Comprehension 1990-92 
Continuous Membership N=2^83 
(plot of NCEs with national percentile change noted) 
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Figure 55 

Andersen Contemporary CAT Reading Comprehension 1990-92 
Continuous Membership N= 68 
(plot of NCEs with national percentile change noted) 





Figure 56 

Bancroft CAT Reading Comprehension 1990-92 
Continuous Membership N= 91 
(plot of NCEs with national percentile change noted) 
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Figure 57 

Field CAT Reading Comprehension 1990-92 
Continuous Membership N=131 
(plot ofNCEs with national percentile change noted) 
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Figure 58 

Wilder Contemporary Reading Comprehension 1990-92 
Continuous Membership N=158 
(plot of NCEs with national percentile change noted) 
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Figure 59 

Holland CAT Reading Comprehension 1990-92 
Continuous Membership N=158 
(plot of NCEs with national percentile change noted) 





Year 



45 






C-2 



Figure 60 

Lyndale CAT Reading Comprehension 1990-92 
Continuous Membership N= 158 
(plot of NCEs with national percentile change noted) 
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The National Center on Education in the Inner Cities 

The National Center on Education in the Inner Cities (CEIC) was established on November 1, 1990 by the Temple 
University Center for Research in Human Development and Education (CRHDE) in collaboration with the University of Illinois 
at Chicago and the University of Houston. CEIC is guided by a mission to conduct a program of research and development that 
seeks to improve the capacity for education in the inner cities. 

A major premise of the work of CEIC is that the challenges facing today’s children, youth, and families stem from a 
variety of political and health pressures; their solutions are by nature complex and require long-term programs of study that apply 
knowledge and expertise from many disciplines and professions. While not forgetting for a moment the risks, complexity, and 
history of the urban plight, CEIC aims to build on the resilience and "positives" of inner-city life in a program of research and 
development that takes bold steps to address the question, "What conditions are required to cause massive improvements in the 
learning and achievement of children and youth in this nation’s inner cities?" This question provides the framework for the 
intersection of various CEIC projects/studies into a coherent program of research and development. 

Grounded in theory, research, and practical know-how, the interdisciplinary teams of CEIC researchers engage in studies 
of exemplary practices as well as primary research that includes longitudinal studies and field-based experiments. CEIC is 
organized into four programs: three research and development programs and a program for dissemination and utilization. The 
first research and development program focuses on the family as an agent in the education process; the second concentrates on 
the school and factors that foster student resilience and learning success; the third addresses the community and its relevance to 
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CEIC’s findings are known, but also to create a crucible in which the Center’s work is shaped by feedback from the field to 
maximize its usefulness in promoting the educational success of inner-city children, youth, and families. 
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